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This document contains the NCCPE’s final response to the KEF consultation that was 
launched in January 2019. We would like to thank the many people who commented on and 
contributed to this document. 


We have also published a Briefing pack about the KEF to translate the consultation document for 
a non-HE audience, and to provide some prompts for a discussion about it: 


If you would like to discuss this response, please contact: nccpe.enquiries@uwe.ac.uk 


Q.4 KEF PURPOSE 


Do you consider that the KEF as outlined will fulfil its stated purposes? 


e To provide universities with new tools to understand, benchmark and improve 


their performance. 


e To provide business and other users with more information on universities. 
e To provide greater public visibility and accountability. 


Strongly 
disagree 


Disagree 


Somewhat 
disagree 


Somewhat 
agree 


Agree 


Strongly 
agree 


No opinion 


To provide universities 
with new tools to 
understand, 
benchmark and 
improve their 
performance. 


To provide businesses 


and other users with 
more information on 
universities. 


To provide greater 
public visibility and X 
accountability. 


Please provide a commentary in relation to your scores above. (400 word limit) 


We see potential in the KEF being used as a strategic tool by institutions to identify 
distinctive features of their KE provision and compare their activity with other 
similar institutions. In the case of Public Engagement, the proposed metric is of 
limited value, and we propose an alternative approach below 

The KEF will provide centralised and standardised information which partners and 
the public more generally could access. However, as currently configured, we aren’t 
confident that the information is sufficiently tailored to the needs of these potential 
audiences, so it is not likely to achieve this objective. We suggest below how the 
content relating to Public Engagement and Local Growth might be configured 
differently to better meet this objective, but we think this will need user testing 
The proposed perspectives provide a broad articulation of the types of activity 
encompassed by KE, and treat these as equally important. This is a positive step, as 
it challenges a widely held view that KE is primarily about commercialisation and 
technology transfer. However, the proposed metrics are largely focused on financial 
value and the rationale for these 7 perspectives is not explained. We think this 
merits further thought, to ensure they best capture the variety of KE activity, and 


provide the answers that external users of the KEF would find valuable. We were 
surprised that a technical rationale for the choice of perspectives was not provided 
in the consultation, as it was for the clustering methodology. 

While we appreciate the intention to minimise burden on HEls, and to utilse existing 
data where possible, there is a significant risk that the KEF is foregrounding what can 
be measured rather than what really matters. The focus in university income in 
most of the perspectives excludes significant types of value generated through KE 
interactions. The provision of narratives for all 7 perspectives would increase the 
burden, but allow vital context to be provided. 

If the intention is to use the KEF to exercise accountability, then it makes sense to 
consider linking it to funding (longer term, once the methodology is proven). 
However, there are risks with doing this given how relatively crude the proposed 
metrics are, and the decision not to assess or evaluate the narrative statements, 
which provide more nuanced intelligence. If it is to be linked to funding, then some 
form of light touch review of these narratives would be necessary. 


5. AIMS AND OVERALL APPROACH OF THE KNOWLEDGE EXCHANGE FRAMEWORK (KEF) 


Overall approach 


The KEF consultation document describes the overall approach as being an annual, 
institutional level, largely metrics driven exercise, although noting that narrative will have an 
important role. More background may be found in the report summarising the 
recommendations of the technical advisory group. Do you consider this overall approach to 
be appropriate? * 


Strongly disagree 


Disagree 


Somewhat disagree 


Somewhat agree | 


Agree 


Strongly agree 


No opinion 


Please provide a commentary in relation to your scores above. (400 word limit) 


We appreciate the considerate effort taken by RE to develop a framework that aims 
to both make use of existing data and reduce burden on institutions. 

However, until the proposed metrics are operationalised through the pilot it is hard 
to judge the robustness and usefulness of these metrics or to anticipate other 
(perhaps unintended) consequences of the proposed approach. 

The exercise is also seriously constrained by the availability of useful metrics. The 
current review of HEB-CI is clearly very significant as it could provide a much better 
long term solution by ensuring more robust and useful metrics are routinely 


captured, which the KEF could benefit from. We would argue strongly that this 
iteration of the KEF should be viewed as a relatively crude ‘demonstrator’ with the 
expectation that it will evolve considerably over the next 3 years. 

Generally, the proposed metrics start from the assumption that income is a proxy 
for excellence. The substantial body of research into knowledge exchange does not 
corroborate this. The literature foregrounds the vital role of sustained partnerships; 
capacity building and ‘absorptive capacity’, and demonstrates that some of the most 
valuable outcomes for partners have a low ‘unit price’. 

Given the above, we welcome the inclusion of narrative elements within the 
framework. However, the fact that these narratives will not be assessed limits their 


value. We suggest how this might be addressed below. We also argue that all 7 
perspectives would benefit from accompanying narratives. 
e External perspectives from HE partners are not currently captured by the KEF. It 
would be interesting to consider how, longer term, a systematic way of capturing 
such feedback might be captured (for instance, using the net promoter score, an 
approach used in other sectors). 


e The limited involvement of partners in developing and shaping the KEF is a serious 
concern. There is a risk that the KEF ends up reflecting what the HE sector thinks is 
important, not what partners need from or value about the sector. We have 
attempted to canvas views from some partner organisations in relation to the public 
and community engagement dimension, but this is a very limited sample. We 


recommend that significant user involvement should be factored into future 


iterations of the KEF 


6. CLUSTERING 


The English higher education sector is very diverse. We therefore propose to create clusters 
of knowledge exchange peer groups. The proposed clusters and clustering approach is 
detailed in the KEF consultation document. Please use the following questions to provide 
your feedback on our proposals. 


Please indicate your degree of support for the following aspects of our clustering 


approach. * 


Strongly 
disagree 


Disagree 


Somewhat 
disagree 


Somewhat 
agree 


Agree 


Strongly 
agree 


No 
opinion 


The conceptual 
framework that 
underpins the cluster 
analysis. 


The variables and 
methods employed in 
undertaking the 
cluster analysis. 


The resulting make- 
up of the clusters, i.e. Xx 
the membership. 


That the overall 
approach to 
clustering helps 
Research England to 
meet the stated 
purposes of the KEF 
and ensures fair 
comparison. 


Please provide commentary on any aspect of your scores above. If relevant please 
incorporate suggestions for alternative arrangements. (400 word limit) 


We welcome the use of clustering in the KEF, linked to institution’s assets and 
capabilities for KE, as it makes the task of comparing very different types of 
institutions more intelligent and fair. It could also allow for interesting intelligence 
to merge over time: for instance, do the different clusters have distinctively different 
approaches to community and public engagement, and if so, why is this? 

The approach taken to developing the conceptual framework seems broadly fair and 
logical. 

We agree with Million Plus that, perhaps inadvertently, the descriptors for Clusters J 
and M are expressed negatively (the use of the word ‘limited’) in particular. It would 
be better to describe all clusters in positive terms, and without implying comparison 
with other clusters. 

We did note that two very significant variables were missing from the clustering 
methodology. 

Firstly, we were surprised that the role of KE professional staff was not identified as 
a variable in the first dimension, ‘Scale and focus of existing knowledge base’. The 
literature makes it clear that these professional roles are a critical factor in 
delivering effective knowledge exchange. Universities invest significantly in these 
roles and this kind of expertise to deliver their KE strategies, for instance by 
employing specialists in public and community engagement, and in other roles to 
manage enquiries, pursue new opportunities, manage relationships, or deal with 
more technical issues such as licensing. It would be useful to explore the relative 
investment HEls make into such specialist roles. 

We were also surprised by a significant omission in the second dimension, ‘Scale 

and focus of knowledge generation’: the proportion of submitting units achieving 
3*/4* impact case studies in the REF. It focuses instead only on the proportion of 4* 
research submitted by individual researchers. Given that the quality of impact 
achieved by HEIs is critically linked to their KE capabilities we are keen to understand 
why this was not incorporated into the methodology, and whether including it 
would have any impact on the clusters. 


7. Perspectives and metrics 


Knowledge exchange covers an extremely diverse range of activity and it is appropriate that 
some HEls will perform more strongly in different areas that align more closely with their 
mission and strategic goals. We have therefore proposed a range of seven perspectives. The 
following questions will seek your views on the number and range of perspectives and 
metrics proposed. 


Perspectives 


e Research partnerships 

e Working with business 

e Working with the public and third sector 
e Skills, enterprise and entrepreneurship 
e Local growth and regeneration 

e IP and commercialisation 

e Public and community engagement 


Taking into account the overall range of perspectives and metrics outlined in the 
consultation document, do you agree or disagree that a sufficiently broad range of KE 
activities is captured. * 


e Strongly disagree 


e Disagree 


e Somewhat disagree 


e Agree 


e Strongly agree 


e No opinion 


Comments: 


e The perspectives provide a relatively helpful categorisation of different types of KE, 
with some provisos noted below. 

e We agree that all seven perspectives should receive equal weighting. Their function 
is to capture the broad range of KE activities being undertaken. Weighting them 
equally will encourage a broad and balanced portfolio of KE activities across the 
sector. 


e However, it would have been helpful if the consultation had provided a rationale for 
why these 7 perspectives were chosen. Compared with the clustering, which is 
robustly explained, the perspectives lack an explicit rationale. This begs the question 
of how they were derived: 

o Do they represent where the HE sector is currently investing its KE 
resources? Do they capture roughly equivalent blocks of investment and 
activity? 

o Alternatively, were they chosen as areas where metrics would be relatively 
easy to identify? 

o Orto represent, broadly, the types of interaction and impact which RE 
believes the HE sector should be attentive to? 

o Do they in any way represent what stakeholders in the system believe should 
be priorities for the HE sector to address — in effect, acting as a 
representation of ‘demand’ and ‘need’ from outside the sector? 

e Our impression is that, currently, they are something of a pragmatic ‘hybrid’ of these 
different motivations. It would help if RE could provide a more robust articulation of 
why they were chosen. 

e Building on the above, we have some concerns that the perspective of ‘working 
with the public and third sector’ encompasses a disproportionately large portfolio 
of types of KE activity, incorporating as it does interaction with the cultural, health, 
local government, voluntary and education sectors, and potentially policy too. We 
wonder if this could / should be broken down further to allow more room for these 
areas of activity to ‘breathe’. 

e We note that social innovation or enterprise has a key role to play in building an 
inclusive economy and the metrics as currently presented do not reflect this. 

e Weare concerned by the lack of focus placed on partnerships between HEIs. 
Developing such collaborations and partnerships makes a critical contribution to the 
overall impact of the sector’s KE performance, and the vitality of the KE system. If it 
is decided that an overarching narrative should be provided by HEls to frame their 
KEF profile, then this might be included as an explicit prompt within this. 


Research partnerships 


Taking into account the range of metrics outlined in the consultation document for this 
perspective, please provided any comments on the balance and coverage of the proposed 
metrics. (400 word limit) 


e As argued above, we think that it could be helpful to include a third measure, to 
capture the extent and vitality of research partnerships with other HEls. 

e While we applaud the spirit of the metric to count numbers of co-authored papers 
we worry this may have unintended consequences, corralling partners into an 
activity that may not be meaningful or valuable to them, or for which they are not 
recompensed. It may skew activity away from other more valuable types of 
interaction. 


There is a significant tension in the KEF as a whole: its goal is to provide a robust, 
national picture and so HEls are clustered nationally. However, many potential 
collaborators with HEls will want to work with HEls that are local to them — so may 
want to compare HEls within a fixed geographical range. It would be useful to 
explore for this perspective, and for others, percentages for each of the proposed 


metrics that are in the local region. This would complement the local growth perspective. 
It is also important to note how activity often ‘spills across’ perspectives — for 
instance, the important contribution that public engagement can make to this 
perspective. If the perspectives are treated as mutually exclusive, some of the 
important vitality of the KE system might be suppressed. 


Working with business 
Taking into account the range of metrics outlined in the consultation document for this 


perspective, please provided any comments on the balance and coverage of the proposed 
metrics. (400 word limit) 


e Nocomment 


Working with the public and third sector 
Taking into account the range of metrics outlined in the consultation document for this 


perspective, please provided any comments on the balance and coverage of the proposed 
metrics. (400 word limit) 


e Nocomment 


Skills, enterprise and entrepreneurship 
Taking into account the range of metrics outlined in the consultation document for this 


perspective, please provided any comments on the balance and coverage of the proposed 
metrics. (400 word limit) 


e Nocomment 


Local growth and regeneration 


Taking into account the range of metrics outlined in the consultation document for this 
perspective, please provided any comments on the balance and coverage of the proposed 
metrics. (400 word limit) 


Note there is a separate question to consider the use of supplementary narrative. 


The definition for this perspective includes the phrase ‘spill-over outcomes that relate to 
local growth and regeneration’ and activities ‘targeted to make a difference locally.....this 
may include social inclusion’. Social inclusion and many other outcomes of local 
regeneration activity (community development, environmental improvement, etc.) cannot 
be captured by a metric focused entirely on income. Additionally, the focus on income 
suggests that universities should be profiting from their local area, rather than vice versa. 


Additional or alternative metrics, such as % FTE involved in local activities, % of 
modules/courses with local placements, % academics on local boards, opportunities for the 
local community to engage with the university, etc., would show a fuller picture. 


Other possible metrics which, over time, could be routinely captured include: 


e We contribute financial, staff and other resources to the ongoing development of 
our Local Industrial Strategy. The estimated value of these resources is £X 


e We have developed in partnership with other stakeholders, including neighbouring 
universities, a transparent and prioritised action plan of practical investments 
selected on their basis of their prospective contribution to the Local Industrial 
Strategy and/or Grand Societal Challenges. The value of financial investments made 
with own resources of our university in the period X to X was £Y 


e We build our staff and financial capacity to attract and then invest external finances 
to part fund these investments and increasingly share this capacity with other local 
stakeholders. The value of external financial resources secured in in the period 
was £X. The cost of our staff capacity to attract and invest these external finances in 
the period was £Y 


e We havea planned approach to the ongoing use of the Social Value Act within our 
procurement policies. We provided in the period X to X training to Y local firms to 
develop their capacity to submit successful tenders in response to our procurement 
programmes. Approximately X% of procured services and goods were bought from 
local firms. 


e We area Living Wage employer. Y/N 


We provide further suggestions of meaningful indicators in our response to the narrative 
question below 


IP and commercialisation 


Taking into account the range of metrics outlined in the consultation document for this 
perspective, please provided any comments on the balance and coverage of the proposed 
metrics. (400 word limit) 


e Nocomment 


Public and community engagement 


Taking into account the range of metrics outlined in the consultation document for this 
perspective, please provided any comments on the balance and coverage of the proposed 
metrics. (400 word limit) 


Note there is a separate question to consider the use of supplementary narrative. 


We have reservations about the proposed metric (time per academic FTE committed to 
events, performances and museums and galleries): 


e These types of activities represent just a subset of the many types of public and 
community engagement which we would want to see reflected in the framework. 
For many HEls these kinds of dissemination activities are just the ‘tip of the iceberg’, 
and don’t capture more interactive and collaborative types of activity 

e HEls often do not have accurate ways of tracking staff time, raising concerns about 
the accuracy of the data 

e The inclusion of events in museums (which need to be ‘owned by the HEP’) could 
unfairly bias the framework in favour of the relatively small number of HEIs who 
own their own museums 


We think a more reliable metric could be provided by the use of a different HE-BCI metric: 
the survey already asks (Q.6): Does your HEP have a strategic plan for public and community 
engagement? HEls are asked to self-report against a five point scale. The existence of a 
strategy is, in our view, a more robust proxy indicator of ‘excellence’ in public and 
community engagement than the proposed metrics of hours dedicated to events. The 
narrative account could require them to justify the ranking they have provided and to 
provide more details of their strategic support. There are of course limitations as this 
indicator relies on self-reporting. However, it is a sensible first step. 


A further step would be to use the linked narrative statement to require more nuanced self- 
reporting against a broader set of indicators, allowing an overall aggregate metric to be 
created. In this way the narrative becomes a critical component in the assessment, rather 
than a de-coupled context setting device. 


HEls could be invited to self-assess against these indicators on a scale of one to four (where 
1 is not in place; and 4 is fully realised). Their overall ‘score’/metric would be the average of. 
their assessments of their progress against the indicators. 


We suggest that the indicators might cluster into four groups, capturing key factors in 
effective support for public engagement: 


e Having a strategic plan and operational commitment to public and community 
engagement 

e Providing community access to your facilities and expertise. 

e Involving communities in your research and teaching 

e Commitment to partnership working and social responsibility 


We elaborate on this in the narrative section below. | 


8. SUPPLEMENTARY NARRATIVE 


We consider that for two perspectives, that on their own, the existing metrics do not 
provide sufficient measure of the scale and variety of activities undertaken by higher 
education institutions (HEIs). 


We intend to work with the sector to develop, where possible, metrics that will capture the 
outcomes derived from all types of knowledge exchange in the future. In the mean time we 
propose to supplement both the Local Growth & Regeneration and Public & Community 
Engagement perspectives by requesting a narrative statement from each provider to set out 
the main strategic goals, activities, outputs and potential outcomes achieved. 


Do you consider it appropriate for HEIs to provide narrative text to support the metrics in 
perspectives that don't currently have fully developed metrics? * 


e Strongly disagree 


e Disagree 


e Somewhat disagree 


e Somewhat agree 


e Agree 


| e Stronglyagree | 


e No opinion 


Public and community engagement narrative 


Overall, is the guidance on the provision of narrative text for this perspective clear. * 


e Strongly disagree 


e Disagree 


e Somewhat disagree 


| © Somewhatagree | 


e Agree 


e Strongly agree 


e No opinion 


Please comment on the proposal to include narrative from HEls for the public and 


community engagement perspective, in particular: - where further clarification is required- 
where refinements could be made- whether there are areas where more consistency across 


HEls could be achieved (400 word limit) 


We propose two possible routes forward. One is to use an alternative HEBCI metric (existence of 


strategy), and to use the narrative to evidence the infrastructure in place to support it. The 


second, to invite HEls to self-assess on a numerical scale against a more extensive set of indicators. 


As a consequence, we suggest the narrative template be adapted to: 


e Focus on indicators representing good practice in engagement, inviting them to self-report 


against these 
e Ensure these indicators address types of support which external partners most value 
e Require that HEls provide evidence and links, to ensure transparency and usability 


We have published a draft template on our website. 


Below is a sample of the kinds of indicators we are suggesting, derived from existing literature: 


1. Providing community access to your facilities and expertise. 


e We have a transparent process for communities to contact the university and to have their 


enquiries dealt with promptly 
2. Involving communities in your research and teaching 


e We provide a clearly signposted and high quality portfolio of learning and engagement 


opportunities for the public, robustly evaluated 
3. Commitment to partnership working and social responsibility 


e We have a strategy in place to direct our efforts to address equality and diversity in our 
interactions with wider society, and processes in place to monitor this 

4. Strategic commitment to public and community engagement 

e There is a strategic and operational plan in place for PE/CE 


We would anticipate providing statements which allow HEls to match their performance against 
levels, as we currently provide in our EDGE tool. We are aware of the risk that self-reporting will 
encourage HEIs to overestimate performance, for reputational gain. If KEF is implemented as a 
self-improvement tool and is not linked to funding, then this risk would be mitigated. We also 
suggest HEls be asked to provide links to evidence to support their self-assessment. A further 
safeguard would be to review a sample of the submissions to monitor activity. 


The benefits of this approach include: 


e — It would focus attention on critical characteristics of high quality KE activity, addressing the 
objective of improving practice 

e If these indicators link to published information and resources / assets it would address 
the objective of supporting external organisations to navigate the HE sector 


In the interim, the HEB-Cl review would allow the development of more robust metrics which 
could be integrated into future iterations of the KEF 


| 


Local growth and regeneration narrative 


Overall, is the guidance on the provision of narrative text for this perspective clear. * 


e Strongly disagree 


e Disagree 


e Somewhat disagree 


| è Somewhatagree — | 


e Agree 


e Strongly agree 


e No opinion 


Please comment on the proposal to include narrative from HEls for the local growth and 
regeneration perspective, in particular: 


e Where further clarification is required 
e Where refinements could be made 


e Whether there are areas where more consistency across HEls could be achieved 
(400 word limit) 


We suggest that a similar approach might be taken to the narrative and metric for this perspective 
as proposed above. A set of indicators could be derived from the literature about effective KE 
practice in this domain to allow the generation of another self-reported metric, and a narrative 
which provides useful context and evidence / links to back it up. 


A particularly useful source for these is the recently published final report of the Civic University 
Commission, which provides a useful typology of roles HEls can play in their cities and regions. We 
recommend that these be used to structure the narrative account in the KEF for this perspective. 


We provide some possible indicators below for four different dimensions: 


Supporting the educational growth of a place: which encompasses the institution's interaction with 
school aged population, and with mature learners, such as adult, community and lifelong learning; 
and to support skills and employment outcomes for local people 


e Our widening participation and ‘outreach’ activity seeks to actively contribute to attainment and 
employability of local populations 

e We take a strategic approach to meeting the skills and development needs of local employers 
and facilitate KE through student placements in local organisations 


Supporting the economic life of a place: which encompasses the institution acting as a model 
employer and its procurement practices, its local ‘convening’ role, and its role as a leading and 
model economic actor 


e Weare a living wage employer (4 or nothing) 
e Our procurement policies activity seek to deliver value to our locality 


Supporting the cultural wellbeing of a place: which encompasses the institution’s participation in 
and contribution to the cultural life of their areas; support for the cultural and creative and heritage 
sector through work with local partners and helping them to contribute to ‘place making’ 


e We maximise opportunities for our staff and students to actively participate in the cultural life 
of our region 

e We have a strategic approach to supporting local cultural and heritage infrastructure, for 
instance museums and public libraries 


In addition, we recommend that HEls be invited to report on their overarching strategic approach 
and investment into this area: 


Strategic investment to maximise our local impact 


e There are partnership agreements in place that have been co-designed with local stakeholders 
that articulate shared targets and goals for your activity 
e There is regular and systematic reporting on the activity and its impact, with agreed KPIs 


The role of further narrative or contextual information 


We welcome responses on what other types of narrative or contextual information would 


be helpful. 


You may wish to consider, for example: 


e Should the HEI or Research England provide other narrative information? 

e How should we use other contextual information, such as information on local 
economic competitiveness described in section 5 of the cluster analysis report? 

e Would other perspectives benefit significantly from further narrative 
information? 

e Would the benefit of adding further narrative information be outweighed by the 
burden of doing so? * 


Strongly 
disagree 


Disagree 


Somewhat 
disagree 


Somewhat 
agree 


Agree 


Strongly 
agree 


No opinion 


Overarching 
institutional 


by the HEI 


statement - provided 


Overarching 
institutional 


statement - provided 
by Research England 


Comments: 


e We could see the benefit of RE providing a narrative section for each of the clusters. 
This could provide an overview of the strengths of each cluster and provide a clear 


way for those outside the sector to gain an appreciation of the diversity of potential 
partners. 

We also think that other contextual data held by RE for individual HEIs could be 
made available to help external organisations better understand the sector and 
potential partners within it. Any such activity should however be conducted in close 
collaboration with such potential audiences for / users of the KEF 

In providing such context, there are a number of factors which impact on KE which 
could be foregrounded. One example concerns location, for instance when an HEI is 
in a rural location. Engagement with SMEs is far easier in clustered spaces (towns) in 
contrast to more spread communities (eg. rural). Similarly ‘public engagement’ is 
made more difficult by having more challenges around access in rural areas. HEls in 
areas with high ‘absorptive capacity’ have a head start over those in areas without 
such capability and infrastructure etc 


We also believe that an overarching narrative statement from HEls should be 
required, just as the TEF requires a provider statement, and the REF an Environment 
return. Intelligent alignment with these statements should be sought. 

This kind of background information is critically important. It provides useful 
contextual information, which would be of value to users of the KEF to help them 
understand the priorities, resources and infrastructure to support KE that is available 
within different HEls and would help them to navigate the data presented for the 7 
perspectives. It also helps to ensure that the KEF focuses attention not just on the 
‘ends’ but the ‘means’ by which the outcomes have been achieved: the internal 
structures and capabilities to support KE (for example, KE offices) and the 
overarching strategic intent and context which would allow users of the KEF to 
interpret their results intelligently. 

We argue earlier in our response that all 7 perspectives would benefit from 
narratives to help users interpret the results. 


9. VISUALISATION 


We have provided further information including example visualisations of the KEF within the 
consultation document. 


Please comment on the presentation and visualisation proposals, for example: 


e Where further clarification is required 

e Where refinements could be made 

e Whether there are areas where more consistency across HEls could be achieved- 
how narratives could be incorporated? 
(400 word limit) 


e In principle, the top level visualisations are good at showing cluster and individual 
institutional strengths. However, they will only be as robust and useful as the data 
affords. It will be important to test these visualisations with ‘real’ data to ensure 
they present a meaningful and fair comparison between HEIs 

e The visualisations are somewhat helpful in terms of identifying quantity of activity 
but do not indicate quality. Visualisations are based on the quantitative metrics and 
do not take into account qualitative information — this is provided alongside rather 
being part of the scoring. The visualisations can therefore be misleading as a 
measure of quality knowledge exchange. 


e The use of radar charts is good at demonstrating how each perspective has an equal 
weighting, something that a bar graph (for example) would not do as well due to the 
order that perspectives would be listed in. 

e It will be important to test how useful provider dashboards would be to a 
prospective partner / collaborator from outside the sector. The information is very 
‘high level’ and may be difficult to interpret meaningfully. 

e As they are currently configured, it is difficult to see how the narratives (if they are 
two A4 sides) can be easily included in the visualisation. Potentially, the cluster 
description section on the left of the page could be adapted to present some 
narrative information. 

e If however our suggestion of including a set of indicators in the narrative is taken up, 
then these indicators (and the HEls self-reported score against them) could usefully 
be incorporated into a dashboard for this perspective, providing a more nuanced 
picture of that HEls activity 

e While the mocked up dashboards provide one useful ‘lens’ for inspecting the KEF 
data, a well-designed database and interface could allow very different user-defined 
dashboards to be created. It would be sensible to involve creative expertise in 
visualisation and user-focused design to contribute to the next phase of 
development. 


10. IMPLEMENTATION 


We will pilot the implementation with a group of HEls as described in the consultation 
document. 


Please provide any comments about the implementation of the KEF. (200 word limit) 


We believe that the timeline proposed to implement the KEF is promising. We welcome the 
pilot exercise 


We are concerned by the relatively low level of consultation that there has been to date 
with organisations outside higher education who are envisaged as potential audiences for 
and users of the KEF. 


11. ANY OTHER COMMENTS 


If you have any other comments, please share them here. (400 word limit) 


We have drawn on various resources and reviews to compile our list of draft indicators for 
public and community engagement. Our sources include: 


e The NCCPE’s ‘EDGE self-assessment tool’, which was itself based on a extensive 
review of existing benchmarking frameworks 
(http://www. publicengagement.ac.uk/support-engagement/strategy-and- 
planning/edge-tool) 

e The NCCPE’s ‘Auditing, Benchmarking and Evaluating Public Engagement’ report, 
which conducted a comprehensive review of benchmarking frameworks 
(http://www. publicengagement.ac.uk/sites/default/files/publication/evaluatingpubl 
icengagement_1.pdf) 

e The NCCPE’s Community Partner Network’s ‘Principles of practice for Community- 
University partnership working’ 

(http://www. publicengagement.ac.uk/sites/default/files/publication/principles_of_ 
practice_for_cups.pdf) 

e AHRC’s ‘Creating Living Knowledge’ report (https://connected-communities.org/wp- 
content/uploads/2016/04/Creating-Living-Knowledge.Final_.pdf) 

e ‘The Common Cause Principles for Community-University partnerships’ (https://cpb- 
eu- 


w2.wpmucdn.com/blogs.bristol.ac.uk/dist/a/358/files/2018/09/CC Enablers Barrie 


rs Poster _final-1f2iunj.pdf 
e The recent TEFCE report ‘Mapping and Critical Synthesis of Current State-of-the-Art 
on Community Engagement in Higher Education’ 


https://ris.utwente.nl/ws/portalfiles/portal/76357596/TEFCE Mapping Critical S 
nthesis.pdf) 


We echo Praxis Auril’s comments that there is lack of clarity about who the KEF is for and how it 
can present complex interactions accurately to policymakers, university management, business 
audiences and other publics in a single tool. Most significantly, the voice of the user is distinctly 
lacking in current discussion about the KEF and this needs to be captured if the exercise is to meet 
the second of its stated aims and provide universities with information on how KE performance can 
be improved. 


We also agree that as institutions consider the next REF and TEF there needs to be a better 
conversation about how these frameworks fit together and work for institutions, their 
employees and external stakeholders. One very positive impact of KEF discussions to date 
has been the coming together of different groups to understand the reach and remit of KE 
within different types of HEI and how it reaches across research and teaching activities at 
the local, regional and international level. This conversation is welcomed and should be 
maintained. 


Annex 1: draft narrative template for Public and Community Engagement 


Narrative report Comments 


1. Institution name 


2. Contact details: The contact details provided should be relevant to the public engagement 
activities of the institution. The information will be published as part of the narrative statement and 
can be a named individual or a generic contact point. 


3. Context: What is distinctive about how your institution approaches public and community 
engagement: 


Internally: how is engagement reflected in your mission and embedded in your teaching 
and research? 


Externally: what are your priorities in terms of place (your geographical reach); people (do 
you focus on particular communities); and outcomes (have you identified priority outcome 
areas)? 


4. Strategic commitment to public and community engagement: Please indicate your progress 
with the indicators below (where 1 is not in place; and 4 is fully realised) 


There is a strategic and operational plan in place for PE/CE as a result 1 Ajea 
of an inclusive process across the whole HEP and with 
external partner input 


There is a budget allocated to PE / CE with explicit and ambitious 1 21|5|44 
targets 
There is regular and systematic reporting on the activity and its 1 A | 8 | a 


impact, with agreed KPls 


Links to evidence 


There is a senior leader with formal responsibility for PE/CE (or that 
responsibility is distributed clearly across several senior staff) 


There are specialist staff employed to provide support and advice 


There are promotion / career pathways to support staff progression 
which are well used 


We enable the public to access and make use of our facilities and 
assets (e.g. sporting and cultural) and to signpost these opportunities 


We have a transparent process for communities to contact 
the university and establish contact with our staff and to have 
their enquiries dealt with promptly and professionally 


We ensure staff and student expertise is accessible to communities, 
and actively support our people to take up voluntary roles in the 
community 


We provide a clearly signposted and high quality portfolio of 
purposeful learning and engagement opportunities for the public 
which are robustly evaluated 


We actively involve the public in our research activities, and provide 
expert support to facilitate this 


1 


We have a systematic and managed approach to partnership working 
with civic, community and cultural partners in our region, and beyond. 
This includes a published set of partnership principles addressing 
issues such as payment, IP, equity and sustainability 


We have a strategy in place to direct our efforts to address equality 
and diversity in our interactions with wider society, and processes in 
place to monitor this 


Annex 2: draft narrative template for Local Growth and Regeneration 


e There is a strategic and operational plan in place for local growth and i | 2 3 4 
regeneration 


e There are partnership agreements in place that have been co-designed with iL | 2 3 | a 
local stakeholders that articulate shared targets and goals for your activity 


e There is regular and systematic reporting on the activity and its impact, with 1 21314 
agreed KPIs 

e There is a senior leader with formal responsibility for local growth and 1 2m Eo | @ 
regeneration (or that responsibility is distributed clearly across several senior 
staff) 

e There is investment in expertise and resources to support effective iL | 2 3 4 


monitoring and evaluation, and a systematic approach to gathering evidence 


Our widening participation and ‘outreach’ activity seeks to actively 
contribute to attainment and employability of local populations 


We have a strategic approach to deploying our staff and student expertise, 
research and other assets to boost overall attainment in our region 


We take a strategic approach to meeting the skills and development needs of 
local employers and facilitate knowledge exchange through student 
placements in local organisations 


We provide a clearly signposted and high quality portfolio of learning 
opportunities for local people at various stages of life, which are robustly 
evaluated 


We seek to actively involve local publics and partners in our research, 
teaching and knowledge exchange activities, and provide expert support to 
facilitate this 


We seek to maximise opportunities for our staff and students to actively 
participate in the cultural life of our region 


We have taken every effort to enable the public to access and make use of 
our sporting and cultural facilities and assets, and to signpost these 
opportunities 


e We make a significant contribution to the vitality and sustainability of local 1 2 | 3 | 4 
cultural and creative industriess 


e We have a strategic approach to supporting local cultural and heritage 1 a5 já 
infrastructure, for instance museums and public libraries 


e Weare a living wage employer (4 or nothing) 1 2A E3 TEA 

e We actively support staff and students to take up voluntary roles in the 1 21/314 
community 

e Our procurement policies activity seek to deliver value to our locality 1 20 E | a 

e We contribute actively to a range of local strategic partnerships (e.g. LEPs) 1 2314 

e We provide significant support to local spin out companies and SMEs and 1 2) 3 | 


help with attracting and retaining inward investors 


e We ensure our estate development plans have maximum impact on local 1 2 | 3 | 4 
place making and economic development 


Centre for 


National 
| i | Co-ordinating 
| Public Engagement 


The National Co-ordinating Centre for Public Engagement (NCCPE) is 
internationally recognised for its work supporting and inspiring universities 
to engage with the public. 


We work to change perspectives, promote innovation, and nurture and 
celebrate excellence. We also champion meaningful engagement that makes 
a real and valued difference to people’s lives. 


The NCCPE is supported by the UK Higher Education Councils, Research 
Councils UK and Wellcome, and has been hosted by the University of Bristol 
and the University of the West of England since it was established in 2008. 
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